1 



Stationary and dynamical properties of information entropies 

in nonextensive systems 



Hideo Hasegawa 







Department of Physics, Tokyo Gakugei University 
Koganei, Tokyo 184-8501, Japan 

(February 2, 2008) 
Abstract 

The Tsallis entropy and Fisher information entropy (matrix) are very important 
quantities expressing information measures in nonextensive systems. Stationary and 
dynamical properties of the information entropies have been investigated in the N- 
unit coupled Langevin model subjected to additive and multiplicative white noise, 
which is one of typical nonextensive systems. We have made detailed, analytical and 
numerical study on the dependence of the stationary-state entropies on additive and 
multiplicative noise, external inputs, couplings and number of constitutive elements 
(N). By solving the Fokker-Planck equation (FPE) by both the proposed analytical 
scheme and the partial difference-equation method, transient responses of the infor- 
mation entropies to an input signal and an external force have been investigated. 
We have calculated the information entropies also with the use of the probabil- 
ity distribution derived by the maximum-entropy method (MEM), whose result is 
compared to that obtained by the FPE. The Cramer-Rao inequality is shown to 
be expressed by the extended Fisher entropy, which is different from the generalized 
Fisher entropy obtained from the generalized Kullback-Leibler divergence in con- 
formity with the Tsallis entropy. The effect of additive and multiplicative colored 
noise on information entropies is discussed also. 
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1 INTRODUCTION 



In the last half century, considerable studies have been made on the Boltzman-Gibbs- 
Shannon entropy and the Fisher information entropy (matrix), both of which play impor- 
tant roles in thermodynamics and statistical mechanics of classical and quantum systems 
PQ-[7]. The entropy flux and entropy production have been investigated in connection 
with the space volume contraction [2]. In the information geometry [8], the Fisher in- 
formation matrix provides us with the distance between the neighboring points in the 
Rieman space spanned by probability distributions. The Fisher information matrix gives 
the lower bound of estimation errors in the Cramer-Rao theorem. In a usual system con- 
sisting of N particles, the entropy and energy are proportional to N (extensive) , and the 
probability distribution is given by the Gaussian distribution belonging to the exponential 
family. 

In recent year, however, many efforts have been made for a study on nonextensive 
systems in which the physical quantity of N particles is not proportional to N [9j [TOj CD] • 
The nonextensivity has been realized in various systems such as a system with long-range 
interactions, a small-scale system with large fluctuations in temperature and a multi- 
fractal system [11], [12]. Tsallis has proposed the generalized entropy (called the Tsallis 
entropy hereafter) defined by [TO] 



where q is the entropic index, p(x, t) denotes the probability distribution of a state x at 
time t, the Boltzman constant k is hereafter unity and \\i q x expresses the g-logarithmic 
function defined by \n q x = {l — x 1 ~ q ) /(q—1). The Tsallis entropy accounts for the nonex- 
tensivity of the entropy in nonextensive systems. In the limit of q — > 1, ln ? x reduces to the 
normal lnx and then S q (t) agrees with the Boltzman-Gibbs-Shannon entropy expressed 



The probability distribution derived by the maximum-entropy method (MEM) with the 
use of the Tsallis entropy is given by non-Gaussian distribution [TTJ, which reduces to the 
Gaussian and Cauchy distributions for q—1 and q = 2, respectively. 




(1) 



(2) 



by 




(3) 
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Many authors have discussed the Fisher information matrix in nonextensive systems 
[13]-[23]. In order to derive the generalizedFisher information matrix G whose components 
are given by [19J 



9ij 



»* = «M*>(^)Pl?)*- 



(4) 



the generalized Kullback-Leibler distance of D(p \ p') between the two distributions p and 
p' has been introduced: 

D(p\p')=K(p\p') + K(p' \p), (5) 



with 



K(p | p') 



p(x) q \hi q p(x) — \ia q p'(x)} dx, 
1 



1 



p(x) q p'(x) 1 q dx 



(6) 



(9-1) 

where p(x) = p(x; {9i}) and {6*j} denotes a set of parameters specifying the distribution. 
In the limit of q — *■ 1, gij given by Eq. (j4]) reduces to the conventional Fisher information 
matrix. It should be remarked that Csiszar [25] had proposed the generalized divergence 
measure given by 



D C (P I p') 



p\x)f 



p(x) 



+ p{x)f 



' p'{x) 



dx, 



^p'(x) J x ' J \p(x) 

where f(x) is assumed to be a convex function with the condition /(l) = 0. For f(p) 
plnp, Eq. yields the conventional Kullback-Leibler divergence [26] given by 



(7) 



Dkl(p I p') 



p{x) In 



p(x) 



+ p'(x) In 



p'(x) 



dx. 



K p'(x) J ' " y ' \p(x) 

Equation (0) for f(p) = {q — ^)~ l {p q —p) leads to the generalized Kullback-Leibler distance 
given by Eqs. (jSJ) and (El). The generalized divergence given by Eq. ([6]), which is in 
conformity with the Tsallis entropy, is equivalent to the a-divergence of Amari [8] with 
q = (1 — a)/2 [271 12H]- The escort probability and the generalized Fisher information 
matrix are discussed in Refs. [191 [20]. The Fisher information entropy in the Cramer- Rao 
inequality has been studied for nonextensive systems [Ml [201 ETJ ■ 

Extensive studies on the Tsallis and Fisher entropies have been made for reaction- 
diffusion systems, by using the MEM with exact stationary and dynamical solutions for 
nonlinear FPE [TBI [141 [T7i US]. These studies nicely unify the concept of normal, super- 
and sub-diffusions by a single picture. 
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The purpose of the present paper is to investigate the stationary and dynamical prop- 
erties of the information entropies in the coupled Langevin model which has been widely 
adopted for a study of various stochastic systems (for a recent review, see [29]). The 
Langevin model subjected to multiplicative noise is known to be one of typical nonex- 
tensive systems [TT]. Recently the coupled Langevin model subjected to additive and 
multiplicative noise has been discussed with the use of the augmented moment method 
[30] which is the second-moment method for local and global variables [311 [32] . We will 
obtain the probability distribution of the nonextensive, coupled Langevin model by us- 
ing the Fokker-Planck equation (FPE) method with the mean-field approximation. We 
have made a detailed study on effects on the stationary information entropies of additive 
and multiplicative white noise, external force, input signal, couplings and the number of 
constituent elements in the adopted model. By solving the FPE both by the proposed 
analytical scheme and by the partial difference equation (PDE) method, we have investi- 
gated the transient responses to an input signal and an external force which are applied 
to the stationary state. 

The outline of the paper is as follows. In Sec. 2, we describe the adopted, iV-unit cou- 
pled Langevin model. Analytical expressions for the Tsallis entropy and generalized Fisher 
information entropy in some limiting cases are presented. Numerical model calculations 
of stationary and dynamical entropies are reported. In Sec. 3, discussions are presented 
on the entropy flux and entropy production and on a comparison between g-moment and 
normal-moment methods in which averages are taken over the escort and normal distribu- 
tions, respectively. Section 4 is devoted to our conclusion. In the Appendix, we summarize 
the information entropies calculated with the use of the probability distribution derived by 
the MEM. The Cramer-Rao inequality in nonextensive systems is shown to be expressed 
by the extended Fisher information entropy which is different from the generalized Fisher 
entropy. We will discuss effects of additive and multiplicative colored noise on information 
entropies, by using the result recently obtained by the functional- integral method [33J. 
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2 Coupled Langevin model 
2.1 Adopted model 

We have adopted the iV-unit coupled Langevin model subjected to additive and multi- 
plicative white noise given by 

dx ' 

= F{ Xi ) + p&{t) + aG{ Xi )rji(t) + h{t), (9) 

at 

with 

Here F(x) and G(x) denote arbitrary functions of x, J the coupling strength, I(t) an 
external input, a and (3 are the strengths of multiplicative and additive noise, respectively, 
and r]i(t) and express zero-mean Gaussian white noises with correlations given by 

(m(t) %(*')> = (ii) 
= M(*-0, (12) 

(»*(*) 6(0) = °- (13) 
We have adopted the mean-field approximation for given by 

/<(*) ~ J^,(t)-Xi(t)] +/(*), (14) 

with 

' " fry ("> 

***(*) = ^E^M*)L ( 16 ) 

where the £7 g [•] expresses the average over the escort distribution to be shown below [Eqs. 

(EDI- (Ell. 



2.2 Fokker- Planck equation 

Owing to the adopted mean-field approximation given by Eq. ( fT4l) . each element of the 
ensemble is ostensibly independent. The total probability distribution of p({xk},t) is 
given by the product of that of each element: 

p{{x k }, t) = Ui pi(xi, t), (17) 
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where the FPE for pi(xi, t) in the Stratonovich representation is given by 
d 



dt 



Pi(.Xi,t) 



d ( f3 2 \ d 2 

[F{Xi) + Ii{t)\Pi{x h t) + — ^Z2Pi(xi,t) 



dx 



2 ) 8xV 



The expectation value of n q {t) is given by 



with 



E q [ Xi (ty 



H q {t) = E q [xi(t)}, 



P iq (xi,t) x™ dx u (m = 1,2) 



where the escort probability distribution P iq (xi,t) is given by 



(19) 

(20) 

(21) 
(22) 



It is noted that fi q (t) and Pi(xi,t) are self-consistently determined from Eqs. ffT4l and 
f|T8|) . The relevant fluctuation (variance) of cr q {t) 2 is given by 



Piq y^i ; 



Pi (•£» , ^)^j 



Qg(t) 

Cjg(t) = / pi(x i7 t) q dXi. 



a q {t) 2 = E q [{x l -^ q ) 2 }. 



When we adopt -F(x) and G(x) given by 

F(x) = 
G{x) = 



—A x, 



(23) 



(24) 
(25) 



where A denotes the relaxation rate, the FPE for p(x, t) is expressed by (the subscript % 
is hereafter neglected) 



d_ 

Iff 



p(x,t) 



a 



\ + J + —]p{x,t) + 



A + J + — x - u(t) 



d_ 

dx 



p(x,t) 



a 



(3 2 \ d 2 



with 



u(t) = Jn q (t)+I(t). 



(26) 



(27) 
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From the FPE given by Eq. (|26|) . the stationary distribution is given by [30j [3H [35] 

/2A + 2J + a 2 ~ 



lnp(x) oc 



oc 



2a 2 



In 



ln(aV + /3 2 ) + F(x), 

,.2 



with 



1 + 



2a 2 

(2A + 2J + a 2 ) 

/? 2 



(2A + 2J + a 2 ) 



s / 2u \ i ( ax 



(28) 
(29) 

(30) 

(31) 

(32) 
(33) 



where the entropic index is given for 1 < q < 3. Equation ( |29i) yields the g-Gaussian 
distribution given by 



p(x) 



x~ 



exp q 



Z~"^ q \ 20 2 



with 



f ( x 2 
J ^ i"2^ 



(34) 



(35) 



where exp Ax) stands for the g-exponential function defined by exp (x) = [1 + (1 — 



q)x 



1/(1-9) 



where 



y for y > and for y < 0. 



Some limiting cases of Eqs. (1341) are examined in the following. 
(1) For a = and /? 7^ (i.e. additive noise only) 

1 



p(x) 



-(l/2of)(x- M ) 2 



which yield 



27T(7 2 
20 2 M 



P 2 (x+jy 

P 2 



2(A + J) 

(2) For a 7^ 0, (3 = 0, (i.e. multiplicative noise only) 



p(x) = — I x I 5 e (x/k) , for u 7^ 



CX X 



for m = 



(36) 

(37) 
(38) 

(39) 
(40) 



with 

r (g-l) _r((3-cy )/(,/-!)) 

(42) 



^ - ^= ^ *"*0, (41) 



(9-1)' 
or or 

where T(x) and G(x) denotes the gamma and Heaviside functions, respectively, and Z q 
diverges for u — 0. For a^O, Eqs. (1311 and (JUJ) yield 

A 4 ? - (44) 



_ g 2 (g-l) 3 a 

" 4(3 — o) { ] 



The distribution given by Eq. (13111) has a peak at a; = k/8 = [x q /q. 

(3) For a ^ 0, |J ^ 0, a = J/x g + / = (i.e. without coupling and external input), 
[301 EH E5] 

= f^iy%fi,jL4V (47) 



g - 1 J \2q-l 2 



which lead to 



fi q = 0, (48) 
2 20 2 /3 2 , . 

It is noted that when we adopt normal moments averaged over the g-Gaussian given by 

E[x(t) m ] = J p(x,t) x(t) m dx, (50) 

in stead of the g-moments given by Eq. (I20p . its stationary variance is given by er 2 = 
E[(x - E[x]) 2 ] = p 2 /2(\ - a 2 ) which diverges at A = a 2 [30]. 

2.3 Tsallis entropy 

With the use of the total distribution of p({xi}) given by Eq. (fTTj) . the Tsallis entropies 
of single-unit and TV-unit ensembles are given by 

sf) = = (i_^f ) , (52 ) 



1 - 




g - 




l - 




g 


- 1 
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with 



-iq 



Eliminating c q from Eqs. (15"TT) and (152]) . we get 



S q N) 



N 



k=l 



(53) 



(54) 



(55) 



where Cj? = N\/(N — k)\k\. Equation (l54"j) shows that the Tsallis entropy is non-extensive 
except for q = 1.0, for which Sjj N > reduces to the extensive Boltzmann-Gibbs-Shannon 
entropy: S{ N) = N S{ 1] . 

Substituting the stationary distributions given by Eqs. (1361) . (|39|) and (PfBT) to Eq. ([1]), 
we get the analytic expression for the Tsallis entropy of a single unit given by 

r 

1 - C n 



°q 



[l + hnW)], 



for a = 0, ^ 



for a 7^ 



with 



i r( 9 «j-i) i r(( 9 + i)/(g-i)) 



(56) 
(57) 



Z q q (qnY 5 - 1 Z q q (g«)(9+l)/(«-l) 



for a ^ 0, /3 = 0, u + (5* 



20 : 



2 \ 1/2 



2| V?-i 



B 



q 



l 



3-g 



71-9 



2'g-l 2; V 2 

for a ^ 0, /3 ^ 0, u = 



(59) 



where B(a,b) stands for the beta function, and Z q in Eqs. (1581) and ( 1591) are given by 
Eqs. (fldl) and (T4T1) . respectively. 

2.4 Generalized Fisher information entropy 

We consider the generalized Fisher information entropy given by 



/ In j)(x \ \ 

'!•! = 1 l P( X ) I QQ J 



(60) 



From Eqs. (fTTj) and ( 1601) . the generalized Fisher entropy for the iV-unit system is given 
by 



g(N) 

Uq 



qE 



d\iap(x) 
06 



(61) 
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E 



8 In pi(xi 



86 



Hi pi(xi)dxi, 



<?E 



86 



Pi(xi) dxi + Ag q , 



q ' 



(62) 

(63) 
(64) 



because the cross term Ag q of Eq. (|63|) vanishes: 

9 In pi (x 



Ag q 



86 



Pi\*Ei) dXi 



d In Pj(xj] 
86 



Pj(xj) dxj, 



q S E y ^ 

9 



(j r 8 r 

q e e ^ y ^ ^ y ^ 



0. 



(65) 

(66) 

(67) 
(68) 



where g q ^ stands for the generalized Fisher entropy in a single subsystem. The generalized 
Fisher information entropy is extensive in the nonextensive system as shown in [27J: 



9i N) = Ngi 1 \ 



(69) 



The probability distribution p(x) obtained by the FPE for our Langevin model is 
determined by the six parameters of A, a, j3, J, I and N. When adopt 6 = I in Eq. (IBTil) . 
for example, we get the generalized Fisher entropy given by 



g q 



q\E 



'dY(x) 
81 



E 



'8Y(x) 
81 



1 



(70) 



where E[-] expresses the average over p(x) [Eq. ( 1501 ]. 

Alternatively we have adopted the generalized Fisher entropy given by 

// 8 In T>(x \ \ ^ 
1-^-1 dx, 



(71) 



which is obtainable for g q with 9 — fi q in Eq. ( 1601) if p(x) is given by the MEM [Eqs. 
( 11021) and (1A10I) ]. Although p(x) derived by the FPE is not explicitly specified by \i q and 
<jg, we have employed Eq. (ITT]) in our following discussion, expecting it is meaningful for 
both cases of the FPE and MEM. Substituting the stationary distributions given by Eqs. 
(I3"9~|) and (14"6"|) to Eq. (ITTj) . we get the analytic expression for the generalized Fisher 
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entropy for N = 1 given by 



9? = (^)=^A for a = 0,0*0 (72) 

)4 
<J(5-l)((y + 2) = % foro^O, (3 = 0, u*0 (73) 

- ((^)Stf^|^. fortt , 0,^0,^0 ,74) 
where ex 2 in Eqs. (!73|) and (ITil) are given by Eqs. (H5l) and f|49l . respectively. 

2.5 Stationary properties 

2.5.1 Calculation method 

The adopted Langevin model includes six parameters of A, a, j3, J, I and N. The 
dependence of the Tsallis entropy and generalized Fisher information entropy on these 
parameters have been studied by numerical methods. We have calculated the distribution 
p(x) by the FPE [Eqs. and (1351)]. and also by direct simulations (DSs) for the 

Langevin model [Eqs. ([9]) and (fTUI) ] with the Heun method: DS results are averages of 
100 trials. 

2.5.2 Model calculations 

Figures [D(a)-[I](c) show three examples of the stationary distribution p(x) for (/, J) = 
(0.0, 0.0), (0.0, 0.5) and (0.5, 0.5) with A = 1.0, a = 0.5, (3 = 0.5 and N = 100. Solid 
curves show the results calculated with the use of the FPE whereas dashed curves those of 
DSs for the Langevin equation: both results are in good agreement and indistinguishable. 
When the coupling strength is increased from J = 0.0 to J = 0.5 with I = 0.0, the width 
of p(x) is decreased because of a decreased 2 in Eq. ( 13T1) . When an input of / = 0.5 
is applied, p(x) changes its position by an amount of about 0.5 with a slight variation 
of its shape: p(x) for (/, J) = (0.5, 0.5) is not a simple translational shift of p(x) for 
(/, J) = (0.0, 0.5). 

In the following, we will discuss model calculations of the dependence on a, (3, I, J 
and N, whose results are shown in Figs. [2], [3], HI [5] and O respectively (dotted curves in 
the frames (a) and (b) in Figs. [2}l5] will be explained in Sec. 3.6.1). 

a dependence 
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First we show a q , S q and g q in Figs. a) , Wfo) , [21(c) and 121(d), respectively, plotted 
as a function of a 2 for I = 0.0 (chain curves), I = 0.5 (dashed curves) and / = 1.0 (solid 
curves) with A = 1.0, (3 = 0.5 and J = 0.0. Figure [21(a) shows that the a dependence of 
/iq is very weak. We note in Fig. [21(b) that for / = 0.5 and / = 1.0, a 2 is linearly increased 
with increasing a 2 though a 2 is independent of a for I = 0.0. Figure [21^c) shows that with 
increasing a 2 , S q is increased with broad maxima at a 2 ~ 0.8 for / = 1.0 and at a 2 ~ 1.5 
for / = 0.5. With increasing a 2 from a 2 = 0, in contrast, g q is decreased for / = 0.5 and 
/ = 1.0 with broad minima, whereas g q is independent of a 2 for I = 0.0. For larger J, S q 
and g q have stronger a 2 dependence. 

(3 dependence 

Figures E^a), [3](b), [3](c) and [3]^d) show fi q , a 2 , S q and g q , respectively, plotted as 
a function of j3 2 for / = 0.0 (chain curves), / = 0.5 (dashed curves) and / = 1.0 (solid 
curves) with A = 1.0, a = 0.5 and J = 0.0. With increasing (3, \i q has no changes although 
a 2 is linearly increased. With increasing (3 2 from (3 2 = 0.0, S q (g q ) is significantly increased 
(decreased). This trend is more significant for / = 0.0 than for / = 0.5 and / = 1.0. 

/ dependence 

The / dependence of fi q , a 2 , S q and g q are shown in Figs. IH(a)-[4](b) for a = 0.0 (chain 
curves), a = 0.5 (dashed curves) and a = 1.0 (solid curves) with A = 1.0. The gradient 
of \x q versus / is slightly larger for larger a. In the case of a = 0.0, a 2 , S q and g q are 
independent of / [see Eqs. (1561) and (1721) ]. With increasing I for finite a, S q is increased 
while g q is decreased. 

J dependence 

We show the J dependence of fi q , a 2 , S q and g q in Figs. [5](a)-[5](b), for / = 0.0 (chain 
curves), / = 0.5 (dashed curves) and / = 1.0 (solid curves) with A = 1.0, a = 0.5, (3 = 0.5 
and iV = 100. We note that \i q is independent of J. With increasing J, a 2 and S q are 
linearly decreased whereas g q is increased. 

iV dependence 

Figure [6] shows the Tsallis entropy per element, S q N>) /N, given by [Eq. (|55|) ] 

= SM-±(N-l)( q -l)(SWf + .., (75) 
for a = 0.0 (dotted curve), a = 0.1 (solid curve), a = 0.5 (dashed curve) and a = 1.0 
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(chain curve) with A = 1.0, f3 = 0.5, / = 0.0 and J = 0.0. Note that for a = 0.0 
(q = 1.0), the system is extensive because S[ N ^ /N = S^'. For finite a, however, it is 
nonextensive: S^/N is more significantly decreased for larger a, though the generalized 
Fisher information entropy g q is extensive [Eq. f l69p ], 

2.6 Dynamical properties 

2.6.1 Analytical method for the FPE 

In order to discuss the dynamical properties of the entropies, we have to calculate the 
time-dependent probability p(x,t), solving the FPE given by Eq. (|26|) . In the case of 
q = 1.0, we may obtain the exact solution of the Gaussian distribution given by 



1 



P {x, t) = , : e -[z-Mt)]72-i(t) 2 ; ( 76 ) 

/2tt a^t) 2 



where Hi(t) and <J\(t) 2 satisfy equations of motion given by 

dfjLi(t) 



dt 

da\{t) 



-XmW+I, (77) 



2 



(// = -2(X + J)a 1 (ty + ^. (78) 

In order to obtain an analytical solution of the FPE for q > 1.0, we have adopted the 
following method: 

(1) Starting from an equation of motion for nth g-moment of -Ejx™] given by 

dE q [x n ] d 
~dt 

q f fdp{x,t)\ ^ n 1 (dc q 



— J P q (x,t) x n dx, (79) 

j(^) p{Xitr i x » dx _L^ Ein (80 ) 

p(x,t) ,-1 dx, (81) 



dc g f I dp(x, t) 



dt *J V dt 

we have obtained equations of motion for fi q (i) (= -Eg [a;]) and u q (t) 2 (= -E g [x 2 ] — E[x} 2 
valid for 0(a 2 ) and 0(/5 2 ), as given by [30] 

dfi q {t) 



dt 
da q {tf 



A/x,(t) + /, (82) 
2(A + J)<r q (t) 2 + a 2 fi q (t) 2 + (3 2 . (83) 



dt 

Equations (J82"j) and (183]) lead to the stationary solution given by 



N = J, (84) 

a 2 = W + P 2 ) _ (« 2 / 2 /A 2 + P 2 ) 
0q 2(A + J) 2(A + J) 
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(2) We rewrite the distribution of p(x) given by Eqs. (!30|) - (l35|) in terms of fi q , a 2 and q, 
as 



p(x) 



(1-9) 



x 



1-9 



(86) 



with 



Y(x) 




(87) 
(88) 



where Z q expresses the normalization factor [Eq. fl35l) ]. In deriving Eqs. f l86|) - fl88|) . we 
have employed relations given by 



a 



/? 2 



2( g -l)(A + J) 
2(A + J) 



a 



2 (? - i)^ 1 



(3-g) 



(89) 
(90) 



which are obtained from Eqs. (1301) . (1841) and (|85il . 

(3) Then we have assumed that a solution of p(x, t) of the FPE given by Eq. (T261) is 
expressed by Eqs. f l86|) - fl88|) in which stationary \i q and a 2 are replaced by time-dependent 
H q {t) and crg(t) 2 with equations of motion given by Eqs. fl82l and fl83|) . 

Dotted curves in the frames (a) and (b) of Figs. 3-6 express the results of stationary 
fi q and a 2 calculated by Eqs. ([HID an d dHS]) for some typical sets of parameters. They 
are in good agreement with those shown by solid curves obtained with the use of the 
stationary distribution of p(x) given by Eq. 

As will be shown shortly, the approximate, analytical method given by Eqs. (1821) . ( IHBl . 
(I86l)-(l88l) provides fairly good results for dynamics of fi q (t), a q {t) 2 and S q (t), and also for 
that of g q {t) except for the transient period. 

2.6.2 Partial difference equation method 

In order to examine the validity of the analytical method discussed above, we have adopted 
also the numerical method, using the partial difference equation (PDE) derived from Eq. 
as given by 

„2\ 



p(x, t + b) 



a 



p(x,t) + I A + J + — j bp(x,t) 



14 




) 



\p(x + a, t) + p(x — a,t) — 2p(x, t)] 



u(t) 



( 



2a 



b 



) 



\p(x + a) — p(x — a)} 



(91) 



with 



u(t) = J n q {t) + I{t) 



(92) 



where a and 6 denote incremental steps of x and t, respectively. 
We impose the boundary condition: 



p(x,t) = 0, 



for I x |> x. 



rn 



(93) 



with x m = 5, and the initial condition of p(x, 0) = po(^) where po(^) is the stationary 
distribution given by Eqs. (134j) and (1351) . We have chosen parameters of a = 0.05 and 
b = 0.0001 such as to satisfy the condition: (a 2 x^ n b/2a 2 ) < 1/2, which is required for 
stable, convergent solutions of the PDE. 

2.6.3 Model calculations 

Response to I{t) 

We apply the pulse input signal given by 



where AI = 1.0 and G(t) denotes the Heaviside function: 0(t) = 1 for t > and zero 
otherwise. Figure [7] shows the time-dependent distribution at various t for A = 1.0, 
a = 0.5, (3 = 0.5 and J = 0.0. Solid and dashed curves express the results of the PDE 
method and the analytical method (Sec. 3.6.1), respectively. When input of AI is applied 
at t = 2.0, the distribution is gradually changed, moving rightward. The results of the 
analytical method are in good agreement with those obtained by the PDE method, except 
for t = 3 and t = 7. 

This change in p(x,t) induces changes in fi q (t), & q (t) 2 , S q (t) and g q (t), whose time 
dependences are shown in Figs. [8](a) and[8]^b), solid and dashed curves expressing the 
results of the PDE method and the analytical method, respectively. By an applied pulse 
input, fi q , a 2 q and S q are increased while g q is decreased. The result for S q (t) of the 
analytical method is in fairly good agreement with that obtained by the PDE method. 
The calculated g q (t) of the analytical method is also in good agreement with that of the 
PDE method besides near the transient periods at t ~ 2 and t ~ 6 just after the input 



I{t) = AI Q(t- 2)9(6 -t), 



(94) 
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signal is on and off. This is expected due to the fact that g q (t) given by Eq. (ITT]) is sensitive 
to a detailed form of p(x,t) because it is expressed by an integration of (dp(x,t)/dx) 2 
over p(x, t), while S g (t) is obtained by a simple integration of p(x, t) q . 

For a comparison, we show by chain curves, the results of the PDE method when the 
step input given by 

I{t) = AI6(t-2), (95) 

is applied. The relaxation time of S q and g q is about 2.0. 

It is noted that input signal for a = induces no changes in S q (t) and g q (t), which 
has been already realized in the stationary state as shown by chain curves in Figs. H](c) 
andSKd). 

Response to A(t) 

We modify the relaxation rate as given by 

A = 1.0 + AA Q(t - 2)6(6 - t), (96) 

which expresses an application of an external force of AF (= — AA x) at 2 < t < 6 with 
AA = 0.5. Figure [9](a) and[9](b) show the time dependence of a 2 S q and g q with a = 0.5, 
(3 = 0.5, / = 0.0 and J = 0.0 for which \i q = 0. Solid and dashed curves express the results 
of the PDE method and the analytical method, respectively. When an external force is 
applied, a 2 and S q are decreased whereas g q is increased. The results of the analytical 
method are in good agreement with those of the PDE method. The relaxation times of 
S q and g q are 0.47 and 0.53, respectively. 



3 Discussion 

3.1 Maximum-entropy method 

In the preceding Sec. 2, we have discussed the information entropies by using the probabil- 
ity distribution obtained by the FPE for the Langevin model. It is worthwhile to compare 
it with the probability distribution derived by the MEM. The variational condition for 
the Tsallis entropy given by Eq. (TjQ) is taken into account with the three constraints: a 
normalization condition and q- moments of x and x 2 , as given by [T4"] [T7] [181 [36] 

1 = J p( x ) dx, (97) 
flq = E q [x\ = J P q (x) x dx, (98) 
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a\ = E q [(x - /i 9 ) 2 ] = J P q (x) (x - fu q ) 2 dx, (99) 
where E q [-] expresses the average over the escort probability of P q (x) given by 

P q (x) = (100) 

Cq 

p(x) q dx, (101) 

the entropic index q being assumed to be < q < 3. After some manipulations, we get 
the g-Gaussian (non-Gaussian) distribution given by [36] 

= l^(-^f]^ ( 102 ) 



Z q e *\ 2va\ 



with 



9 



2 



(103) 



(X - figf 



Z q = J exp ? I 2 I dx, (104) 



2^ 2 \ 1/2 „/l 1 1 



£ -, - - , for 1< q < 3 (105) 

9-1/ V 2 9 - 1 2/ 

2K<r lt for g = 1 (106) 

2,a 2 \ 1/2 A 



9 



n ; B o> 1 + 1 • for < g < 1 (107) 

V 1 - 9/ V 2 1 ~ 9 / 

In the limit of g — > 1, in Eq. fllQ2p becomes the Gaussian distribution: 

V27ra! 

The probability distribution given by Eq. ( 11021) derived from the MEM is different from 
that of Eq. (134j) obtained by the FPE although both expressions are equivalent for 



H q = I = J = with va 2 q = <p 2 . Note that the former is defined for < q < 3 while the 
latter is valid for 1 < q < 3. 

A comparison between the probability distributions obtained by the FPE and MEM 
is made in Figs. [HD and [£U Figure fTDTa) shows the probability distributions calculated 



by the FPE of the Langevin model for a = 0.0, 0.5, 1.0, 1.5 and 2.0, which yield (g, a 2 ) = 
(1.0,0.125), (1.222,0.25), (1.667,0.625), (2.059,1.25) and (2.333, 2.125), respectively, with 
fi q = 1.0 for / = 1.0, A = 1.0, = 0.5 and J = 0.0 [Eqs. (El and (jSSj)]. Figure Mb) 
shows corresponding distributions calculated by the MEM with the respective parameters 
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of q, fi q and a q . For a = 0.0 (q = 1.0), both distributions of the FPE and MEM are 
Gaussian centered at 1.0. For q ^ 1.0, p(x) of the FPE becomes asymmetric 

with respect to x = \i q while that of the MEM is still symmetric. The peak position 
of p[x) of the MEM is at x = fi q = 1.0 independent of q while that of the FPE moves 
leftward with increasing a. It is noted that p(x) of the FPE for cr ^ and (3 = given 
by Eq. fl39|) has a peak at x = \x q jq. 

Figure [TlTa) shows p{x) of the FPE for various inputs of / = 0.0, 0.5, 1.0, 1.5 and 
2.0, which yield (fi q , tf) = (0.0,0.125), (0.5,0.25), (1.0,0.625), (1.5,1.25) and (2.0,2.125), 
respectively, for A = 1.0, a = 1.0, f3 = 0.5 and J = 0.0 [Eqs. and (1851)]: corresponding 
p(x) of the MEM with the respective parameters of q (= 1.667), fi q and a q are plotted 
in Fig. [TTTfb). For n q = 0.0, both the distributions agree. Although centers of both 
distributions move rightward with increasing their profiles and peak positions are 
different between the two distributions. We note that the magnitude of p(x) at x < 0.0 
of the FPE is smaller than that of the MEM for fj, q ^ 0.0. 

The information entropies calculated with the use of the distribution given by Eq. 
( 11021 ) are summarized in the Appendix. One of the advantages of the MEM is that its 
distribution is explicitly specified by the parameters of (#i, 6' 2 ) = ( / u (? , a^) while that of the 
FPE is given in an implicit way [cf. Eqs. fl86l)- (!88l) ]. We may discuss the upper bound of 
estimation errors by the Cramer-Rao inequality, which is shown to be expressed by the 
extended Fisher entropy [Eq. (1A19j) ] but not by the generalized Fisher entropy [Eq. (1A6j) ]. 

In order to discuss the dynamics within the MEM for q ^ 1.0, we have once tried 
to obtain an analytic solution of its distribution p(x,t), assuming that it is given by Eq. 

(una): 




p( x > = ex P 



(X - (J, q (t))' 

2ua q (t) 2 



(109) 



where the g-dependent coefficient A q is determined from Eqs. f ll05l) -( fT07|) . and equations 
of motion for fi q (t) and cr q {t) 2 are derived so as to meet the FPE after Refs. [T71 US] . 
Unfortunately, we could not uniquely determined them: we got two equations for dfi q (t) / dt 
and three equations for da q (t) 2 /dt which are mutually not consistent (except for q = 1.0). 
This implies that the exact analytic solution of the FPE is not given by Eq. (11091) . Indeed, 
the exact solution for (3 = J = in Eq. fT26|) does not have a functional form given by 
Eq. (HTJ9]) 
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3.2 Entropy flux and entropy production 



It is interesting to discuss the entropy flux and entropy production from the time derivative 
of the Tsallis entropy given by 



dS q (t) 
dt 



Q 



Qf + Qa + Qm 



p(x,t) 



,-i ( dpjx, t) ' 
{ dt 



dx, 



with 
Qf 

Qa 

Qm 



q / p(x,t) 



'dF(x) 



' c?q 



dx 
p(x,t) q I 



dx + q ( q -i)J P (x,ty( d -^^) 



d\iap(x,t).' 
dx J 

dlnp(x, t) 



dx 



dx. 



G(x) 



(110) 
(111) 

F(x) dx, (112) 
(113) 



'dG(x)\ d 2 G{x) 



dx 



dx 2 



G{x) 



dx. 
(114) 



Here Qf denotes the entropy flux, and Qa and Qm stand for entropy productions due to 
additive and multiplicative noise, respectively. 

By using the stationary distribution given by Eq. (|4"6"|) . we get Qp, Qa and Qm in the 
stationary state with I = J = (i.e. without couplings and external input): 



2 \ 1/2 



Qi 



M ( 2q g 

Xq(q - 1) / 2a 



bI 1 



2 q-1 2 



2 \3/2 



Qa 
Qm 



°IZ\ \q-\ 
P 2 q ( 2^ 



2 „ / 0~2 \ 3 / 2 



2a\Z\ U-l 



ct 2 g / 2cr 



5 



\2' g- 1 2 / 
'3 1 3\ 



(115) 
(116) 



2 \ 5/2 



2<7*Zf \q-\J 
a 2 ( 2a\ \ 1/2 



B 



5 1 



1 



2Z£ U-l 



B 



2 q-1 2 / 
1 1 1\ 



2' q- 1 2 I ' 



(117) 



where Z q is given by Eq. (H7|) . Equations (|115)) - (|117p satisfy the stationary condition: 
+ Qa + = 0. 

It is worthwhile to examine the limit of a — > (q — > 1.0), in which Eqs. (11 101) . 

dUSD-dZ!) yield 



d5i(t) 



<9p(a;, t) 
dt 



\np(x, t) dx, 



:n8) 
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= Qf + Qa, 



(119) 



with 



Q F = I p(x,t) (^pj dx, (120) 
a 2 \ f , . /<9hi£>(x, t) N ~ 



With noticing the relation: lim|^|_ i . 00 [r(2; + a)/T(z)z a ] = 1 [38J, we may see that Eqs. 
( 11201) and (TT2T1) lead to Q F = —Qa = -A and dS 1 /dt = in the limit of g -> 1. 

In the opposite limit of (3 — > 0, Eqs. (I115l) - (lll7p yields that each of Qf- Qa and Qm 
is proportional to 1/(3^^ and then divergent in this limit, though Qf + Qa + Qm = 0. 
It is noted that Q A = A for a -»• and (3 -»• El i]. 

We present some model calculations of Qf, Qa and in the stationary state, which 
are shown in Fig. [T2]as a function of a for (3 = 0.1 (dashed curves), /3 = 0.5 (chain curves) 
and /? = 1.0 (solid curves). We note that Q F < and + Qm > 0. With increasing a, 
Qf is decreased in the case of (3 = 0.1, while it is increased in the cases of (3 = 0.5 and 
1.0. Bag |4j showed that Qf is always decreased with increasing a which disagrees with 
our result mentioned above: Eqs. f 1 11 5 f) - ( IXTTT) are rather different from Eqs. (36) and (37) 
in Ref. [I] where non-Gaussian properties of the distribution is not properly taken into 
account. 

3.3 q- moment and normal- moment methods 

In Refs. [301132], we have discussed equations of motion for normal moments of [i (= E[x]) 
and a 2 (= E[(x — /u) 2 ]) [Eq. (|50|) ] in the Langevin model with J = 0, as given by 

d -f - -(x-$)m + i. (-) 

da{t) 2 



dt 



2(X-a 2 )a(ty + a 2 fi(ty + (3 2 . (123) 



These equations of motion are rather different from those for the g-moments of \x q and 
a 2 given by Eqs. fl82l and flHBl . Indeed, Eqs. (11221) and (I123j) yield stationary normal 
moments given by 

" = (A^W (124) 

° = 2(X-a 2 ) ' (125) 
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which are different from the stationary g-moments of fi q and a 2 given by Eqs. (I84p and 
(!85|) . and which diverge at A = a 2 /2 and A = a 2 , respectively. 

The time dependence of fi(t) and c(t) 2 becomes considerably different from those of 
H q {t) and cr g (t) 2 for an appreciable value of a. Figure [131(a). (b), (c) and (d) show some 
examples of fi q (t), o~ q (t) 2 , and o~(t) 2 , respectively, when a pulse input given by Eq. 
(JMj) is applied with a = 0.2 (chain curves), a = 0.5 (dashed curves) and a = 0.8 (solid 
curves). Although fi q (t) is independent of a, a q (t) 2 , fi(t) and a(t) 2 are much increased at 
2 < t < 6 for larger a. In particular, <r(t) 2 is significantly increased because of the a 2 
term in Eq. (T123D . 

3.4 Effects of colored noise 

We have so far considered additive and multiplicative white noise. In our recent paper 
[33] , we have taken into account the effect of colored noise by employing the functional- 
integral method. We have assumed the Langevin model subjected to additive (x) and 
multiplicative ((f)) colored noise given by 

^1 = ^Xx(t) + X (t) + x(t)<P(t) + I(t), (126) 
at 

with 

= —be® - £{t)], (127) 
at r a 

= -±[<f>(t)-ar,(t)), (128) 
at T m 

where r a and (3 (r m and a) express the relaxation time and strength of additive (mul- 
tiplicative) noise, respectively, and £ and rj stand for independent zero-mean Gaussian 
white noise. By applying the functional-integral method to the Langevin model given 
by Eqs. fll26l) - fll28p . we have obtained the effective one- variable FPE, from which the 
effective Langevin model is derived as 

dx(t) 



dt 

with 



-Xx(t) + (5 f (t) + d(t) x[t) r](t) + I(t), (129) 



? ^ ark)' < 130 > 

2 

«W 2 = T, ?, w / \ i • (131) 

W [1 + Tm I(t)/n(t)] V ^ 
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Here fi(t) is given by Eq. (11221) with a = a, from which a is determined in a self-consistent 
way. 

In the stationary state where /i = //(A — a 2 /2) given by Eq. (11241) with a = a, we 
get a from Eq. H 1 3 1 p : 

2 

n a , 
« = TT, T, ^v7> (132) 



[l + r m (A-a 2 /2)] 
1 



1 + Ar m ) - J(l + At™) 2 - 2r m a 2 



(133) 



We get an approximate expression given by [33] 



a 2 



a 



2 



t for r m a 2 /2(l + Ar m ) 2 < 1 (134) 

i^l + Ar m ) 

which is shown to be a good approximation both for r m <C (1/A, 2/ a 2 ) and r m ^> 
(1/A, a 2 /2A 2 ) (33]. Equations (I130p and (11341) show that effects of additive and multi- 
plicative colored noise are described by a 2 and (3 2 which are reduced by factors of (1 + Ar a ) 
and (1 + Ar m ), respectively, from original values of a 2 and f3 2 . 

The r a dependence of S q and g q is plotted in Fig. [T4Ta) and fT4Tb) with r m = 0.0 for 
/ = 0.0 (chain curves), / = 0.5 (dashed curves) and I = 1.0 (solid curves) with A = 1.0, 
a = 0.5 and (3 = 0.5. We note that with increasing r a , g q is much increased for smaller I 
whereas S q is much decreased for smaller I. The dependence of S q and g q on r a may be 
understood from their (3 dependence shown in Figs. [31(c) and[^d). The r m dependence 
of S q and g q is plotted in Fig. [T4T a) and [T4Tb) with r a = 0.0 with A = 1.0, a = 0.5 and 
P = 0.5. With increasing r m , S q (g q ) is decreased (increased) for / = 0.5 and / = 1.0 
while no changes for / = 0.0. These behavior may be explained from Figs. [2](c) and[2|d) 
showing the a dependence of S q and g q . 

4 Conclusion 

We have discussed stationary and dynamical properties of the Tsallis and Fisher entropies 
in nonextensive systems. Our calculation for the iV-unit coupled Langevin model sub- 
jected to additive and multiplicative noise has shown the followings: 

(i) the dependence of S q and g q on the parameters of A, a, f3, I, J and N in the coupled 
Langevin model are clarified (Figs. [2][6]), and 

(ii) dynamical properties are well described by the analytical method for the FPE proposed 
in Sec. 2.6.1, which shows that the relaxation times in transient responses of S q and g q 
to a change in A are short (r ~ 0.5) while those in / are fairly long (r ~ 2). 
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The difference between the parameter dependence of S q and g q in the item (i) arises from 
the fact that S q provides us with a global measure of ignorance while g q a local measure 
of positive amount of information pQ. 

We have calculated the information entropies also by using the probability distribution 
derived by the MEM, from which we get the followings: 

(iii) p(x) derived by the MEM is rather different from that of the FPE for fi q ^ (Figs. 
[TU1 and [TT1) . for which the information entropies of the MEM are independent of fi q while 
those of the FPE depend on n q {i.e. I), and 

(iv) the Cramer-Rao inequality is expressed by the extended Fisher entropy [Eq. ( 1A19I) 
which is different from the generalized Fisher entropy [Eq. (1A6j) ] derived from the gener- 
alized Kullback-Leibler divergence [Eq. flBJ]. 

The item (iv) has not been clarified in previous studies on the Fisher entropies in nonex- 
tensive systems [13j-[2"3]. 

The Langevin model has been employed for a study of a wide range of stochastic 
systems [29J. Quite recently, the present author has proposed the generalized rate-code 
model for neuronal ensembles which is described by the coupled Langevin-type equation 
[39| HO] . It would be interesting to discuss the dynamics of information entropies in such 
neural networks, which is left for our future study. 
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Appendix: The Maximum-entropy method 

By using the probability distribution given by Eq. (11021) derived by the MEM, we 
have calculated the information entropies, which are summarized in the Appendix. 

Tsallis entropy 

With the use of Eqs. (CQ) and ( 11021) . the Tsallis entropy is given by 




for q ^ 1 



for q = 1 



(Al) 



(A2) 
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with 



1 (2va, 



2\ 1/2 



z$ W-i 



for 1 < g < 3 



1 /2z/cx 2 \ 1/2 /l g \ 

5 I -, + 1 , for < g < 1 



z\ VI -g, 



2' 1-g 



which yield 



c 9 = i/ Z^ 1 " 9 . for < g < 3 



(A3) 
(A4) 

(A5) 



Here Z g for < g < 1 and 1 < g < 3 are given by Eqs. (1105)) and (1107)) . respectively. 

Generalized Fisher entropy 

The distribution p[x) given by Eq. (1102)) is characterized by two parameters of 
62) = Cq)- By using Eqs. (jlj) and (11021) . we obtain the component of the general- 
ized Fisher information matrix G given by 



9ij 



qE 



d\np(x)\ (d\np(x) 



d6 r 



89, 



qE[(X t - E[X x ]){Xj - ElXj})], for i,j = 1,2 



with 





(x - ^ q ) 2 \ 


exp 9 ^> 





(A6) 
(A7) 

(A8) 



where E[-] denotes the average over the g-Gaussian distribution of p(x) whereas E q [-] 
stands for the average over the escort distribution of P q (x). Substituting the probability 
given by Eq. (fT02]) to Eq. (jA"6|) . we get 

/' / O In /;( ./• ) \ ~ 

9u 

j \ ofi q ) 

2 




dx 

2q \ ^(5, (^Zi) + |, 



,^ 2 (g-l)/ B( 



1 1 



i\ ' 



2' (9-1) 



2g 



3 1 



2' (1-9) 



-1) 



^ 2 (l-g)y 1 +1 



for 1 < g < 3 

for g = 1 

for < g < 1 



(A9) 
(A10) 
(All) 
(A12) 

(A13) 
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which yield 



0ii 



oi 



for < q < 3 



A similar calculation leads to the (2,2)-component given by 



922 



q I p(x) \ dlnP f ) <7.r. 



do* 



' 3-g 



for < q < 3 



The generalized Fisher information matrix is expressed by 





(3-9) ] > 
4a* 



whose inverse is given by 



G" 1 



^ 







(3-9) 



In the limit of q — 1, the matrix reduces to 

4 

^ 



for q — 1 



(A14) 



(A15) 
(A16) 



Extended Fisher entropy: Cramer-Rao inequality 

Next we discuss the Cramer-Rao inequality in nonextensive systems. For the escort 
distribution given by Eq. (11001) which satisfies Eqs. (1981) and ( 199|) with 



1 = E q [l] = J P q (x) dx, 



we get the Cramer- Rao inequality [TJ [16l [201 EI] 

V > G _1 . 



(A17) 



(A18) 



Here V denotes the covariance error matrix whose explicit expression will be given shortly, 
and G is referred to as the extended Fisher information matrix whose components are 
expressed by 



9ij 



E n 



dlnP q (x)\ fdlnP q (x) 



36, 



for i,j = 1,2 



(Xi — E q [Xi])(Xj — E q [Xj\) 



(A19) 
(A20) 
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with 



d 

X i = qq[Q lnp(x)], 
= qik-ElXi]), 



Xi being given by Eq. (1A8j) . Note that g^ is different from g^ given by Eq. 
for q = 1.0. The (1,1) component of G is given by 

2" 



I d\nP q (x) 



2q 2 



( d \ap(x) 



dx 

3 9 



2' (g-1) + 2 



dx, 



yo\{q-\)) 5(1,^-1)' 



2g 2 



5(3 _9 



2' 



1) 



,^ 2 (i-?)/5(i(T^) + i) 



for 1 < q < 3 

for q = 1 

for 1/2 < g < 1 



which lead to 



0ii 



<K? + i) 



(3-g)(2g-l)(7f 
Similarly, the (2,2) component of G is given by 



for 1/2 < q < 3 



#22 



/ <91nP g (x) \ 2 ' 
(9 + 1) 



for 1/2 < q < 3 



4(2g-l)o} 

The extended Fisher information matrix G is expressed by 



</(<?+!) 



whose inverse is given by 



(3-g)(2<z-l)<T= 





(3-q)(2g-l) CT 2 





(q+i) 



--i 



9(9+1) 






4C2g-lVg 
(9+1) 
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A calculation of the component (uy) of the covariance error matrix V leads to 

V 




(5-3g) 

In the limit of q = 1, the matrices reduce to 



r -i I ° q 



4 > 



for q = 1 



Chain and solid curves in Fig. ITST a) express the g dependence of Vn/a 2 and l/pucr^, 
respectively. When q is further from unity, 1/gn is much decreased and it vanishes at 
q — 1/2 and 3. The lower bond of i>n is expressed by the Cramer- Rao relation because it 
is satisfied by g\\. 

v n = — > — . for 1/2 < q < 3 (A31) 
9u 9n 

Chain, dashed and solid curves in Fig. [ToT b) show v^/crq, l/#22Cg and l/^Vq, respec- 
tively. It is noted that u 2 2 diverges at q — 5/3. The following relations hold: 

— > w 22 > forl/2<g<l (A32) 

#22 #22 

v 22 > J- > — ■ for 1 < q < 5/3 (A33) 

#22 #22 

Equation (IA32I) means that 1/(722 cannot provide the lower bound of t>22- Equations 
f)A31j) - flA33l) clearly show that the lower bound of V is expressed by the extended Fisher 
information matrix G, but not by the generalized Fisher information matrix G. 
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Figure 1: (Color online) Stationary distribution p(x) for (J, J) = (0.0, 0.0), (0.0, 0.5) and 
(0.5,0.5) with A = 1.0, a = 0.5 and (3 = 0.5, calculated by the FPE [Eqs. flM} and (1551)] 
(solid curves) and by direct simulations (DSs) for the coupled Langevin model [Eqs. ([HD 
and (fTUj) ] (dashed curves). 

Figure 2: (Color online) The a 2 dependence of (a) fi q , (b) a 2 , (c) S q and (d) g q for / = 0.0 
(chain curves), / = 0.5 (dashed curves) and / = 1.0 (solid curves) with A = 1.0, (3 = 0.5 
and J = 0.0. Dotted curves in (a) and (b) express the analytical result given by Eqs. (1M1) 
and ([85D for / = 1.0. 

Figure 3: (Color online) The /3 2 dependence of (a) fi q , (b) a 2 , (c) S q and (d) g q for I = 0.0 
(chain curves), / = 0.5 (dashed curves) and / = 1.0 (solid curves) with A = 1.0, a = 0.5 
and J = 0.0. Dotted curves in (a) and (b) express the analytical result given by Eqs. (1841) 
and (J8"5) for I — 1.0. 

Figure 4: (Color online) The / dependence of (a) fi q , (b) a 2 , (c) S q and (d) g q for a = 0.0 
(chain curves), a = 0.5 (dashed curves) and a = 1.0 (solid curves) with A = 1.0, (3 = 0.5 
and J = 0.0. Dotted curves in (a) and (b) express the analytical result given by Eqs. (1511) 
and (JESD for a = 1.0. 

Figure 5: (Color online) The J dependence of (a) fi q , (b) a 2 , (c) S q and (d) g q for I = 0.0 
(chain curves), / = 0.5 (dashed curves) and I = 1.0 (solid curves) with A = 1.0, a = 0.5, 
(3 = 0.5 and iV = 100. Dotted curves in (a) and (b) express the analytical result given by 
Eqs. flU and (1531) for I = 1.0. 

Figure 6: The iV dependence of the Tsallis entropy per element, S^/N, for a = 0.0 
(dotted curve), a = 0.1 (solid curve), a = 0.5 (dashed curve) and a = 1.0 (chain curve) 
with A = 1.0, (3 = 0.5, / = 0.0 and J = 0.0. 

Figure 7: (Color online) The time-dependent probability distribution p(x, t) when an 
input pulse given by I{t) = AI Q(t - 2)0(6 - t) is applied with AI = 1.0, A = 1.0, 
a = 0.5 and (3 = 0.5: solid curves express the results obtained by the PDE method and 
chain curves denote those by the analytical method described in Sec. 2.6.1. Curves are 
consecutively shifted downward by 0.25 for a clarity of the figure. 

Figure 8: (Color online) The time dependence of (a) fi q (t) and cr q (t) 2 and (b) S q (t) and 
g q (t) for an input of I{t) = AI G{t - 2)6(6 - 1) with AI = 1.0, A = 1.0, a = 0.5, f3 = 0.5 
and J = 0.0. Solid curves denote the results obtained by the PDE method and dashed 
curves those obtained by the analytical method described in Sec. 2.6.1. Chain curves 
denote the results of the PDE method for an input signal given by I(t) = AIQ(t — 2), 
results of g q and \i q being divided by a factor of ten. 
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Figure 9: (Color online) The time dependence of (a) cr q (t) 2 and (b) S q (t) and g q (t) for 
\(t) = 1.0 + AA G(t - 2)6(6 -t): AA = 0.5, a = 0.5, f3 = 0.5, I = 0.0 and J = 0.0. Solid 
curves denote the results obtained by the PDE method and dashed curves those obtained 
by the analytical method described in Sec. 2.6.1, results of g q being divided by a factor 
of ten. 



Figure 10: Probability distributions p(x) calculated by (a) the FPE and (b) MEM for 
(a,q,a 2 ) = (0.0,1.000,0.125), (0.5,1.222,0.25), (1.0,1.667,0.625), (1.5,2.0591.25) and 
(2.0,2.333,2.125) with I = 1.0, A = 1.0, (3 = 0.5, and J = 0.0, figures in parentheses of 
(a) denoting a values. 



Figure 11: Probability distributions p(x) calculated by (a) the FPE and (b) MEM for 
(H q ,tf) = (0.0,0.125), (0.5,0.25), (1.0,0.625), (1.5,1.25) and (2.0,2.125), respectively, 
with A = 1.0, a = 1.0, {3 = 0.5 and J = 0.0. 



Figure 12: (Color online) The a dependence of entropy flux (Qf), and entropy produc- 
tions by additive noise (Qa) and multiplicative noise (Qm) for (3 = 0.1 (dashed curves), 
P = 0.5 (chain curves) and (3 = 1.0 (solid curves) with A = 1.0, / = 0.0 and J = 0.0. 



Figure 13: The time dependence of g-moments of (a) fi q (t) and (b) <J q (t) 2 , and those of 
normal moments of (c) fi(t) and (d) c(t) 2 with a = 0.2 (chain curves), a = 0.5 (dashed 
curves) and a = 0.8 (solid curves) for an input given by I(t) = AI @(t — 2)0(6 — t) with 
AI = 1.0, A = 1.0, (3 = 0.5 and J = 0.0. The vertical scale of (b) is different from those 
of (a), (c) and (d). 



Figure 14: The r m dependence of (a) S q and (b) g q with r m = 0.0 and the r m dependence 
of (c) S q and (d) g q with r a = 0.0: / = 0.0 (chain curves), / = 0.5 (dashed curves) and 
/ = 1.0 (solid curves) with A = 1.0, a = 0.5 and f3 = 0.5. 



Figure 15: (Color online) The q dependence of (a) Vn/a 2 (= 1/gna 2 ) (chain curve) 
and 1/gua 2 (solid curve), and (b) V22/c q (chain curve), l/g22cr q (solid curve) and \jg22<j\ 
(dashed curve): gij and are elements of the generalized and extended Fisher information 
matrices, respectively. 
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